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DESCRIPTION 



PCT/US99/01647 



IMAGE ACQUISITION AND IMAGE ANALYSIS 
OF BIOLOGICAL MATERIAL 

5 

This invention relates to automated image acquisition and image analysis systems. 
In particular it relates to ^paratus and methods for the high-throughput acquisition and 
analysis of images of fluorescent or chemically stained biological materials. 

10 Background of the Invention 

Many procedures require the analysis of signals derived fiom biochemical reactions. 
For example, cell-based assays have been developed or adapted to generate chromogenic or 
fluorogenic signals vAdch can be detected by hotomultipliers or imaging devices. Also, 
fluoroimmunoassay techniques employ fluorochromes attached to antibodies to detect the 

15 presence of specific biological moieties. Analyzing the resulting signals derived firom these 
moieties con5)letes the assay. Another example is the use of Green Fluorescent Protein (GFP) 
to detamine tiansfection or transformation efficiency in genetic engineering without banrting 
the living cell. Also, cellular assays are done to determine the efficacy of antibiotic 
developments, or in the use of molecular probes to investigate biological processes. Thus, 

20 signals fiom fluorescently stained samples are generated in connection with biological, 
biomedical, and immunoserological applications. There are several current methods for 
analyzing such signals. 

For example, fluorescent plate readers detect light transmitted fiom wells in a 
microtiter plate. According to this tedmique, hght is collected serially or simultaneously fiom 

25 each well in the microtiter plate and directed via a fiber optic cable or lenses to a 
photomultiplier. A badcgroxmd level of fluorescence is determined by averaging light fiom 
the ensemble of unlabeled wells. Wells that exceed the background level of Ught give positive 
results. Because plate readers mtegrate over a large number of individual wells or across the 
entire microtiter plate, they cannot detect single cells. Instead, fluorescent plate readers 

30 usually require a minimum level of approximately 200 stained cells for successfiil signal 
detection within any particular well in the microtiter plate. Although plate readers do not 
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provide microscopic resolution of individual cells, linearity over a limited dynamic range may 
be observed. 

Fluoroimagers op^te similarly, but instead of passively collecting the light they scan 
and image the microtiter plate with an array of CCD sensors. Because of the typically low 
5 spatial resolution of these fluoroimagers, a number of cells may occupy a given pbcel and 
cannot be separately imaged. Thus, like fluorescent plate readers, fluoroimagers usually 
cannot detect smgle cells within a sample. In particular, quantitation of GFP expression levels 
poses some difficulties because the heterogenous expression levels and the low quantum yield 
of the protein produces relatively small amoxmts of fluorescence per active cell. The utility 

10 of fluoroimagers or fluorescent plate readers for quantitation of transfection efficiency or 
expression is therefore rather limited, requiring the use of fluorescence-activated cell sorters. 

A type of flow cytometer commonly refened to as a fluorescence-activated cell sorter 
(FAGS), can detect single transgenic GFP ceils. However, FACS machines require 
trypsinization to disassociate adherent cells because the sorter processes cell suspensions. 

15 Trypsinization may dainage the ceUs and affect the ceUular assay accuracy. FACS machines, 
vMle having greater sensitivity than fluorescent plate readers or fluoroimagers, do not possess 
the high throughput of these analyzers because they analyze only a single cell at a time and 
require the time consuming trypsinization procedure. 

Automated analysis of images from a camera-moimted fluorescent microscope 

2 0 provides the single cell sensitivity of a FACS device without the need for trypsinization and 
with much greater throughput Price et al. (U.S. Pat No. 5,548,661) describe one such system 
analyzing specimens on microscope slides. However, it is difficult to culture cells on glass 
slides as compared to microtiter plates, and numerous glass slides are necessary to duplicate 
experiments performed on a single microtiter plate. Thus, there is a need for an automated 

25 image acquisition and image analysis system which combines the sensitivity of FACS 
machines with the high throughput of fluorescent plate readers and fluoroimagers without the 
need for trypsinization. 

Summary of the Invention 
30 It is an object of the invention to perform quantitative cell assays, such as proliferation 

assays, toxicity assays, ELISAs, and quantitation of transfection experiments and reporter 
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gene expression. For example, reporter gene expression is used to screen delivery vehicles 
or to test the effects of drugs on transcription and translation. Non-radioactive reporter genes, 
sudi as Green Florescent Protein (GPP) and luciferase have been developed for this purpose. 
It is a further object of the invention to classify samples into "positive" or "negative" 
5 classes. For example, the invention will detect the number of cells going through S-phase as 
compared to the total number of cells in the sample. The number of living cells may be 
determined by the number of Syto 13 stained cells, the number of total cells may be 
determined by ethidiimi homodimer staining. 

In one embodiment of the invention, images of samples contained in a plurality of 
1 0 wells may be obtained from a camera mounted to a microscope by scanning the plurality of 
wells with a stage controller. Preferably, the plurality of wells comprises 10 or more wells, 
particularly preferred are 96 wells. The images are digitized and analyzed according to their 
features such as absorbance, fluorescence intensity, or morphology in order to image and 
count individual cells. 

1 5 The advantage of the curroit invention in one ^bodiment lies in the feet that the final 

signal outpxit is binary. In non-imaging devices signals are integrated. A composite signal 
is derived from specific staining, unspecific staining and the background. Furthermore, 
differential staming results in differential output For counting purposes, Ms may not be 
ideal- In one embodiment, the instrument described below does not differentiate between 

2 0 cells stained to different degrees because of its binary output In another embodiment, it is 
not necessary to resolve individual cells but instead quantify cells by merely digitizing the 
obtained image pixels and establishing a threshold value based upon pixel intensity. 

Other objects and features of the invention are illustrated by the following description 
and claims. 

25 

Brief Description of the Drawings 

FIG. 1 is a diagrammatic representation of one embodiment of the invention. 
FIG. 2 is a block diagram of software components in the invention according to one 
embodiment of the invention. 
30 HG. 3 is a flow chart representation oftheiniage acquisition process according to one 

embodiment of the inventiorL 
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FIG. 4 is a flow chart representation of the image analysis process according to one 
embodiment of the invention. 

FIG. 5 A illustrates an image collected by the invention prior to image analysis. 

FIG. 5B illustrates an image collected by the invention after image analysis. 
5 FIG. 6 is a chart illustrating growth analysis by counting at various magnifications 

according to one embodiment of the invention. 

FIG. 7 is a chart illustrating growth analysis by area at various magnifications 
according to one embodiment of the invention. 



10 Detailed Description of the Embodiments of the Invention 

The present invention relates to U.S. Provisional Application No, 60/073,023, filed 
on January 29, 1998, which is hereby incorporated by reference in its entirety. The present 
invention further relates to a system that provides automated image acquisition and image 
analysis of biological material contained in a plurality of wells such as those of a nucrotiter 

15 plate. Images of samples contained in the wells are magnified and captured. The captured 
images are then analyzed. More specifically, in one embodiment, the images of samples in 
a microtiter plate can be obtained fiom a camera mounted to a microscope such as an inverted 
epifluoresc^ce microscope. Images can be obtained fiom any or all of tiie plurality of wells 
by scanning the microtiter plate with a mechanism such as, for example, a stage controller. 

20 To facilitate analysis, the images can be digitized. The final images can be analyzed 
according to their features such as, for example, absorbance, fluorescence intensity, or 
morphology. 

An illustration of a preferred embodiment of the invention is shown in FIG. 1. As 
discussed above, the samples are preferably provided in a plurality of wells, such that images 

2 5 fix)m a plurality of samples can be acquired and analyzed. In a preferred embodiment, the 

plurality of wells is implemented utihzing a microtiter plate 20. According to the illustrated 
embodiment, an inverted epifluorescence microscope 10 provides magnification of samples 
contained in microtiter plate 20. 

An illumination source 60 illuminates the samples. In one embodiment, illumination 

3 0 source is a high-pressure mercury lamp 60. 

Filtering techniques can also be utilized to accommodate multiple chromophores or 
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fluorophores. For example, as illustrated in FIG. 1, appropriate filter cubes 15 with 
interference filters and dichroic minors are used in conjimction with the microscope 10 to 
accommodate multiple chromophores or fluorophores. 

To obtain images fix)m the plurality of wells, microtiter plate 20 is moved relative to 
5 the imaging device. In one embodiment, microtiter plate 20 is mounted on a motorized 
stage 30, and the motorized stage is moved in the x, y and z directions. As illustrated in FIG. 
1, in this embodiment, stage 30 is operated by an XYZ stage controller 40. Preferably, 
software in a computer 80 directs movement of the stage 30 by stage controller 40 in order 
to scan the wells in microtiter plate 20. A raster-type scan pattern is ordinarily suitable to 
1 0 provide coverage of each well, although alternative scan patterns can be implemented as well. 

In one embodunent, commCTcialiy available software such as, for example. Stage Pro 
by Mediatech can be used to direct stage controller 40 to generate the scan pattern and then 
position the microtiter plate 20 to sequentially bring each desired well within the objective 
field of microscope 10. 

15 As would be qjparent to one of ordinary skill in the art after reading ^s description, 

alternative techniques can be utilized to control the position of microtiter plate 20 relative to 

microscope 10, including techniques which position microscope 10 while maintaining the 

position of microtiter plate 20 fixed. 

A cam^ 50 connected with microscope 1 0 records the images. Suitable cameras can 
20 include, for example, the Hitachi HVC20 CCD camera. As would be apparent to one of 

ordinary skill in the art after reading this description, other types of cameras may be used 

without departing from the general principles of the invention. 

As described above, digitization of the captured image facilitates analysis in one 

embodiment. In the embodiment illustrated in FIG. 1, a frame grabber 90 is used to digitize 
25 the c^tured analog image. Frame grabber 90 can be implemented using, for example, the 

commercially available Imagraph Imascan which provides a resolution of 640 by 480 pixels 

per frame. 

The RBG image can be converted to a monochrome image without loss of relevant 
infonnation (fluorescence intensity) and compressed to conserve storage space. In one 
3 0 embodiment, the digitized images are stored as a monochrome JPEG file with 25% 
compression. As would be apparent to one of ordinary skill in the art after reading this 
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description, compression formats other than JPEG can be utilized. Experimentation with the 
JPEG compression format indicates that beyond 25% compression the image degradation 
affects the assay accuracy. 

A monitor 70 is provided in the illustrated embodiment to allow an operator to 
5 supervise image quality, although b one embodiment, the invention can run without human 
intervention. Filenames can be given to each successive image to identify the date, time, and 
position of the analyzed well in microtiter plate 20. 

FIG. 2 illustrates is a diagram illustrating functionality utilized to implement the 
invention according to an example embodiment; in one embodiment, this functionality is 

10 implemented using software components. Stage module 100 provides the functionality to 
operate stage controller 40 so as to initially position a well of microtiter plate 20 within the 
objective field of microscope 20, autofocus the image, and complete the scan pattern of the 
well. This functionality can be implemented using commercially available software such as, 
for example. Stage Pro by Mediatech. Because microtiter plates 20 are typically 

15 manufectured with precise tolerances with the cells adhering to the well surfece, autofocus can 
normally be implemented with a high level of accuracy and accomplished feirly rapidly. 
However, in alternative embodiments, manual focusing can be implemented if necessary or 
otherwise desired. As discussed above, in a preferred embodiment the captured imz^es are 
digitized and compressed. Accordingly, a compressed digital file, such as, for example, a 

2 0 JPEG file 1 1 5 is produced. 

Image processing module 120 performs fte image analysis on the captured images in 
file 115. Specifically, in one embodiment, im^e processing module 120 analyzes spectral 
features contained in the JPEG files 115 representing the digitized images. In one 
embodiment, inaage processing module 120 identifies and counts cells or organelles that 

2 5 satisfy predefined requirements. The total number of such identified objects is calcxilated and 

saved in a data file 160 together with a positional number corresponding to the location of the 
well. In one embodiment, data file 160 is an ASCII file. 

Although this processing can be implemented using a specialized digital image 
processor, satisfactory results can be obtained from commercially available image processing 

3 0 software such as, for example. Image Pro software ruiming on a general purpose computer. 

As would be apparent to one of ordinary skill in the art after reading this description, other 
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image processing software modules can be implemented to process the digitized images. 

A graphics module 140 processes the information contained in the data files 160 to 
provide the results of the analysis. Preferably, graphics module produces a numerical and 
graphical representation of the number of identified cells per well. Averages and standard 
5 deviations of replicate determinations can be automatically calculated and displayed. 

FIG. 3 illustrates an example process by which image acquisition can be performed 
by stage module software 100 according to one embodiment of the invention At step 200, 
the stage is initialized to a starting position. 

The scanning pattern is defined at step 210. Although a raster-type scan pattern 
10 provides satisfactory results^ it would be apparent to one of ordinary skill in the art after 
reading this description that other scanning patterns may be appropriate for a particular sample 
and plurality of wells. The stage is then positioned according to the defined scanning pattern 
at step 220. 

After positioning the stage, autofocusing step 230 occurs. Because microtiter plates 
15 are manufactured to close tolerances with cells typically adhering to the surface of the wells, 
autofocusing at step 230 normally requires little change in the Z position of the stage. 

The image is then captured and digitized by a frame grabber in step 240. The pixel 
levels of the digitized image are converted to a grey scale m step 250 and stored in 
compressed JPEG file 115 in step 260. 

2 0 Decision diamond 270 determines whether the scan is complete. Preferably, this step 

determines Aether thm are any wells remaining to be scanned. If the scan is not complete, 
steps 210 through 270 are repeated to capture theimage fiom the next well, otherwise the 
module exits at step 280. 

FIG. 4 illustrates an example process for unage analysis performed by the image 
25 processing software module 120. The analysis begins with a definition of coxmt criteria at 
step 300. In step 3 10, a search for JPEG files 3 10 is conducted. The appropriate image file 
3 10 is loaded at step 320. 

In the ilixistrated embodiment, the image is digitally filtered at step 330. Digital 
filtering dihances the image by, for example, image segmentation using watershed techniques. 

3 0 Image segmentation allows the analysis of clustered nuclei by determining the edges between 

adjacent nuclei. Although satisfactory results are obtained from the watershed technique. 
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Other filtering or image segmentation techniques can be used. 

After enhancement of the image, the definition of the count criteria determines the 
counting of the desired objects at step 340. Criteria which can be utilized to determine 
whether an object exists and should therefore be counted can include, for example, object 
5 intensity, object size and the shape of the object 

For example, in assaying live rat smooth cells stained with Syto 1 3, a test to detennine 
the presence of the cell nucleus is normally effective. Because the nucleus is typically 
brighter than the background, a test for mtmsity identifies likely cell nuclei. In addition, since 
the cell nucleus typically occupy a given number of pixels at a known magnification, a test 
10 for bright objects occupying an appropriate number of pbcels can be used to help eliminate 
false identifications. Finally, a cell nucleus is conmionly of a rounded shape, and thus a test 
that the X and Y dimensions are within a given tolerance of one another also helps to 
eliminate false positives. Although the combination of these three criteria can be xised to 
accurately identify cell nuclei, it would be apparent to one of ordinary skill in the art after 
1 5 reading this description that alternative testing criteria can be utilized. 

In step 350 it is determined whether there are additional files remaining to be 
analyzed. If there are remaining files, steps 320 through 350 are repeated, otherwise the 
module exits at step 360. 

20 Example 

The above-described techniques are now furftier illustrated by way of a 
Using an embodiment of the invention as illustrated in FIG. 1, live rat smooth cells were 
disposed in a microtiter plate 20 and stained with Syto 13. In filter cube 15, the ^ropriate 
fluorescein filters were installed. FIG. 5 A is a representation of an im^e obtained before 

2 5 digitization and analysis by the invention. As discussed earlier, the cell nuclei are brighter 

than the background, are of similar size, and are rounded. Thus, the count criteria shown in 
FIG. 4 can be used to identify the cells accurately. 

FIG. 5B illustrates the image after analysis. The cells are counted and indicated by 
their individual number. 

3 0 Although this example illustrated a single fluorochiome analysis, it would be apparent 

to one of ordinary skill in the art after reading this description, alternative embodiments of the 
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inventioD can be inq)leinented to analyze multi-label systems. For example, live rat smooth 
cellsstained with Syto 13 and elhidiimihomodimer presents such a syst^ First, an analysis 
with the invention illustrated in FIG. 1 adapted with a fluorescein filter set 15 is conducted. 
Then an analysis with the invention illustrated in FIG. 1 adapted with a rhodamine filter set 
5 15 is performed. For illustration purposes, the resulting images can be overlayed using 
standard image management software such as, for example, Photoshop. Intact, live cells are 
detected in the initial run with the fluorescein filter set, whereas dead or dying cells are 
detected with the rhodamine filter set 

In another example, the above-described techniques were employed to perform a 

1 0 sensitive, ratiometric proliferation assay. This assay is based on the observation that mitotic 
cells are noticeably more spherical than non-mitotic adherent cells. In order to visualize the 
more-spherical mitotic cells, calcein-AM (Molecular Probes, OR) is employed. Calcein-AM 
is a membrane-permeable, non-fluorescent compound which is converted by cytoplasmic 
esterases into a highly fluorescent, membrane-impermeable dye. Mitotic cells not only have 

15 a distinct sh^e, they exhibit significantiy more fluorescence than their non-mitotic 
counterparts because the height of non-mitotic cells is significantiy smaller than in mitotic 
cells. Therefore, the non-mitotic cells will have less calcein molecules in any given area 
segment Because of the shape and fluorescent difiierences, the above-described invention 
may use sh£5>e and fluorescence intensity to distinguish cell populations. 

2 0 The prolifCTation assay sensitivity is greafly enhanced by the addition of nocodazole 

v^ch depolymOTzes miCTOtubules. In the presence of nocodazole, cells entering mitosis will 
undergo the usual morphological change (become more spherical), but they will not be able 
to complete cell division. Thus, all cells entering mitosis will get trapped as mitotic cells, 
exhibiting round cell bodies with bright calcein fluorescence. 

2 5 Experiments have shown proliferation assays performed by the present invration have 

single cell sensitivity. These proliferation assays are as least as sensitive as 3H-thymidine 
incorporation techniques with the advantage of being non-radioactive. It is ratiometric 
because both mitotic and non-mitotic cells are quantified. This is advantageous should the 
initial cell number vary due to treatment or experimental error. Counting cells by 

3 0 conventional metiiods (such as a Coulter counter or MTT-assay) would have only produced 

a two-fold increase in signal strength over a 24 hour period. A much larger signal increase 



• 
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is observed with the calcein-AM proliferation assay. Non-imaging devices, such as plate 
readers, are not suitable to perfoim this assay. Nocodazole-arrested cells may also be detected 
after fixation and staining of cells with DNA-binding dyes such as CyQuant GR (Molecular 
Probes, OR). 

5 As discussed earlier, fluorescent plate readers cannot resolve mdividual cells. 

Nevertheless, despite their limited resolution, such plate readers exhibit linearity over a 
limited dynamic range. This suggests that it may not be necessary to resolve individual cells 
using the above-described invention. To investigate this hypothesis, MRC-5 cells were grown 
for 1 -5 days and subsequently stamed with CyQuant GR (Molecular Probes, OR). Plates were 

1 0 scanned using lOX, 4X, and 2X lenses. The resulting images were analyzed using the above- 
described cell imaging techniques in order to count individual cell nuclei. As expected, FIG. 
6 illustrates that with decreasing magnification, the observed number of individual cell nuclei 
deoieases. A seventeen-fold increase in cell numbers over a four day period is measured using 
a lOX magnification. Instead, if only a 4X magnification is used, the same cell culture 

15 appears to produce approximately a fourteen-fold increase. Moreover, if only a 2X 
magnification is used, the same cell culture appears to produce approximately a ten-fold 
increase in population. Notice that the smallest increase is observed with the Ascent 
fluorescence plate reader which observes only an approximate seven-fold increase in cell 
population. 

2 0 Turning now to FIG. 7, results firom using the above-described invention when only 

pixels with an intensity greater than an automatically deteraiined threshold are quantified. In 
this embodiment, the threshold was determined using convaitional ImagePro software. Pixels 
are assumed to be nonnally distributed about a mean "bright" and a mean "dark" value. The 
threshold is set at the minimum value between the two gaussian distributions. Atthoug^i at the 

2 5 lower magnifications, individual cell nuclei are not resolved, nevertheless; this embodiment 

of the invention illustrates remarkable similarity between the low and high resolution images. 
At a 1 OX power magnification, an qjproximate seventeen-fold increase in cell population was 
observed. The 4X and 2X power measurements detected an approximate 16-fold and 15-fold 
increase. In contrast, an Ascent plate reader measuring the same cell culture detected only an 

3 0 approximate seven-fold increase in cell population. Using this embodiment of the invention 

greatly increases processing speed. For example, scanning a microtiter plate acquiring one 
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image per well at a 2X magpificadon takes less than four 

sixty minutes at a lOX magnification because significantly fewer flames of data are required. 

While this invention has been particularly shown and described with reference to 
preferred embodiments thereof it will be understood by those skilled in the art that various 
changes in fomi and details may be made therein without departing from the spirit and scope 
of the invention defined by the z^pended claims. 
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Claims: 



1. An apparatus for analyzing images of samples contained in a plurality of wells 
comprising: 

5 a microscope for viewing the samples; 

a camera for obtaining infiages from the microscope; 

a stage controller for moving the plxirality of wells in order to scan samples contained 
therein; and 

an image processor for classifying samples according to a specific criteria based }xpon 
10 spectral features. 

2, The apparatus of claim 1 wherein the image processor is a general purpose 
computer. 

15 3. The apparatus of claim 1 wherein the image processor classifies samples 

according to their image intensity. 

4. The apparatus of claim 3 wherein the image processor fiirther classifies 
samples according to their size. 

20 

5. The apparatus of claim 4 wherein the image processor fiirther classifies 
samples according to their roundness. 

6. The apparatus of claim 5 wherein the camera is a CCD camera. 

25 

7. The apparatus of claim 1 wherein the plurality of wells comprises at least 10 

wells. 



8. 

3 0 wells. 



The apparatus of claim 1 wherein the plurality of wells comprises at least 96 




10 
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9. A method of analyzing images of samples contained in a plurality of wells, 
comprising: 

scanning the samples by moving the plurality of wells relative to a camera; 
acquiring images of the samples with the camera; and 

classifying tiie acquired images of the samples according to a specific counting criteria 
based upon spectral features using an image processor. 

10. The method of claim 9 wherein the acquired images of the samples are 
magnified. 

1 1 . The method of claim 9 wherein the classifying step classifies samples 
according to their image intensity. 

12. The method of claim 1 1 wherein the classifying step fiirther classifies samples 
according to their size. 

13. The method of claim 12 \^erein the classifying step fiirther classifies samples 
according to their roundness. 

14. The method of claim 1 0 vy^CTein the camera is a CCD camera. 

15. The method of claim 1 1 wherein the image processor is a general pmpose 
computer. 



1 6. The method of claim 1 2 wherein the samples in the microtiter plate are living 

25 cells. 



15 



20 



1 7. A method of analyzing images of samples contained in a plurality of wells, 
comprising: 

scanning the samples by moving the plurality of wells relative to a camera; 
3 0 acquiring images of the samples with the camera; and 

classifying the acquired images of the samples according to a specific counting criteria 
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produced by a first fluorochrome using an image processor; 

classifying the acquired images of the samples according to a specific counting criteria 
based upon spectral features produced by a second fluorochrome using an image processor, 
and 

combining the classifications in order to perform an image analysis based upon the 
first and second fluorochromes. 

18. A method for analyzing images of samples contained in a plurality of wells, 
comprising: 

acquiring an image of two or more samples contained in Ae plurality of wells; 
and 

classifying the samples according to a specific counting criteria based upon spectral 
features using an image processor. 

19. A method of analyzing images of samples contained in a plurality of wells, 
comprising: 

acquiring an image of samples contained in two or more of said plurality of wells; and 
classifying the samples according to specific features of said captured images. 
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